Fast Semantic Relatedness: WordNet: : Similarity vs Roget's Thesaurus

A Measure of Semantic Relatedness (MSR) automatically determines how close two words are in meaning. MSRs are used in such Natural Language Processing (NLP) problems as word-sense disambiguation or text summarization. To solve such problems may require millions of relatedness scores, but MSR run-time, clearly a major concern, has rarely been considered in NLP research. To evaluate an MSR, one often assigns relatedness scores to word pairs and measures the correlation with human-assigned scores. The WordSimilarity-353 test collection [1] is a known evaluation set of 353 word pairs with given relatedness scores. Spearman’s correlation can be calculated, while Fisher’s r-z transformation can be used to measure significance. We evaluate run-time performance of eleven MSRs previously evaluated with respect to correlation [2]. Resources like WordNet and Roget’s Thesaurus have often been used to create MSRs. Ten MSRs are implemented in the WordNet::Similarity package [3], and one MSR uses the 1911 Roget’s Thesaurus [2]. Roget’s MSR calculates relatedness between two word-senses (appearances of a word in the Thesaurus) in constant time after finding them in an index. WordNet-based MSRs vary in complexity. A few could be implemented to run comparably fast to Roget’s MSR, but WordNet::Similarity seldom does it. Our comparison of two popular MSR packages aims to inform researchers and developers who work on time-sensitive NLP applications. BODY The fastest & best-correlated WordNet MSRs, respectively, take 82 & 182 times longer than Roget’s MSR, yet all are statistically equivalent.